Publish or perish! In this game, you and your opponents will be working on papers to publish. Of course, significant statistics greatly increase your chance to publish, but beware of methodological terrorists checking your p-values or using Bayesian statistics against you! Luckily, questionable research practices or outright Fraud might help you, as long as you are not caught by open science! As always, publication is mostly random and you have to be very luck with the reviewers. The first player who publishes two papers (you have to replicate findings after all) wins!
The following game pieces are needed:
The section “print your own game” below explains how to print out your own game.
Statcheck is a program that checks reported statistical tests to be accurate, and is used in this game. To run statcheck, you either need to install statcheck in R and use R:
statcheck::statcheck("F(..., ...) = ..., p < 0.05", OneTailedTests = FALSE,pEqualAlphaSig = FALSE,OneTailedTxt = FALSE)
or (recommended) use the Statcheck: The Game web app at https://sachaepskamp.shinyapps.io/statcheck/ (also usable on mobile devices)! Important rule: while playing Statcheck: The Game, you may only consult statcheck (or other programs / statistical textbooks / etcetera) when the Statcheck! card is played.
To set up the game:
For example, for a two-player game the board would look like:
Each player rolls the die; the player who rolled the highest starts. In the case of a tie, the tied players roll again until the tie is broken.
You can do the following in your torun:
As common in science, the induction, deduction and testing phases do not need to be played in order and can be repeated as many times as you want per turn. You may not play after attempting to publish, however.
For example, player 1 may start her turn by drawing a card and playing a statistical test:
Her paper is now worth 2 points. Next, player 2 decides to play the card Bayes Factor on player 1’s significance test:
Now, player 1’s paper is only worth 1 point.
At the end of your turn, you may try to publish your paper. To do so, roll the 8-sided die.
For example, player 1 plays another statistical test:
her paper is now worth 3 points and she takes a shot at publishing. She rolls a 6 and does not publish, and draws the reviewer card Lost on the editor’s desk. Player 2 may play twice now.
The 1. Test, 2. Statistic and 3. p-value cards are always played together, and represent adding a statistical test to your paper. Because of publication bias, statistically significant (p < 0.05) results are worth 2 points while non-significant results are only worth 1 point.
The Statcheck! game can be used to test someone’s statistical test using statcheck. You can use the web-app at https://sachaepskamp.shinyapps.io/statcheck/ for this purpose (mobile friendly). For example, player 2 decides to play statcheck on the second significance test played by player 1:
This result is not consistent:
And thus the statistical test is discarded:
Bayesian statistics is a form of magic in which researchers can use “Bayes Factors” to make their own non-significant findings more interesting, or another’s significant findings less interesting. This is reflected by the Bayes Factor card, which can be played on your own and your opponents significance tests. For more information on Bayesian statistics, see the Bayesian Spectacles blog!
Hypothesizing after results come in (are played on the board) is a valuable technique to make your paper more publishable. After all, you totally did expect that non-significant finding!
As is common in academia, not pulling some 60-hour work weeks will severely make you lag behind your opponents. When facing a strong deadline (e.g., your opponent is likely to win next turn), you may even need to work harder, at the risk of burning yourself out!
This card can be used to place 1. Test and 2. Statistic of a played statistical test face-down on the board. This way, you can get make life harder for those pesky methodological terrorists aiming to statcheck your work! Extra rule: you may also choose to play a statistical test from your hand, immediately placing 1. Test and 2. Statistic face-down.
You can choose to cycle the Fraudster card (play it to draw a new card) at no cost. As long as you hold the Fraudster card, you are a fraudster! You can play the Fraudster card then immediately after you fail to publish and before drawing a review card to re-roll the publication die. Being a fraudster thus greatly improves your chances of publishing! Beware of Open Science though!
As in the real world, open science is the natural counter to a Fraudster and researchers working Behind the Pay-wall.
The worst reviewer you can get! Drawing this reviewer will cause you to completely start from scratch on your paper.
Not the worst review, but you will need to put some substantive work in your paper to make it publishable. It will not get accepted in its current form!
Not the worst review, but you will need to put some substantive work in your paper to make it publishable. It will not get accepted in its current form!
Too long; did not read!
Just keep waiting for that e-mail. It is coming any moment now! Aaaaany moment!
Because your peers really objectively think you should cite their work! This card acts as a separate statistical test that cannot be removed or statchecked. It is only removed when the player it is given too publishes his or her paper.
sample(1:8, 1) in R to roll the die